Toward the Autonomous Acquisition of Robot Skill

The design and coordination of independent specialized skill units (often called action primitives) is fundamental to modern robotics. However, a robot that must act in a complex environment over an extended period of time should do more than just use existing skills: it should learn new skills that increase its capabilities and facilitate later problem solving. Although robots exist that can learn a skill given a reward function and hand-engineered state space, none exist today that display truly autonomous skill acquisition. Reinforcement learning [1] is a natural fit for the robot skill acquisition problem, and with the recent development of the options framework [2] provides a principled approach to hierarchical learning and planning and an appropriate model for a robot skill unit (an option) that captures the essential elements required for control. However, although a great deal of research exists on autonomously acquiring skills and skill hierarchies in reinforcement learning, that work has mostly focused on small, discrete domains. In this paper we outline our work on acquiring skill hierarchies in continuous reinforcement learning domains.